Skip to content

Conversation

robertky
Copy link
Member

Closes #7

What changed

  • Adds db:seed:large, which creates a deterministic synthetic album catalog in batches of 1,000.
  • Supports ALBUM_COUNT, defaulting to 100,000 albums, without keeping the full catalog in memory.
  • Keeps the small curated db:seed for normal development and documents both root-level commands.
  • Adds generator tests and records local PostgreSQL performance evidence in the README.

Verified locally

  • Server typecheck and tests (22 tests), repository lint, client tests and both builds pass.
  • A fresh PostgreSQL 16 database migrated successfully; the generator produced 100,000 albums, 5,000 artists and genre relations in 8.59 s.
  • Representative full-text search, combined filters/sorting and a complete searchAlbums request all completed well below one second.

Follow-up

  • Repeat the documented performance check on the group VM after the API deployment is available, since its PostgreSQL version and hardware differ from the local test environment.

@robertky robertky requested a review from a team as a code owner October 15, 2026 18:03
Copy link
Member

@evenkkl evenkkl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, the generator is clean, deterministic and keeps memory flat. I checked out e8701a4 and format, lint, typecheck, tests and both builds pass for me.

Two things before merging, both about the performance evidence, since that's what #7 is really for:

  1. Commit the performance check. The README says to repeat the check on the VM, but the EXPLAIN (ANALYZE, BUFFERS) queries and the searchAlbums request you timed aren't in the repo, so whoever runs it on the VM can't measure the same thing. Could you add them, for example as server/scripts/perf-check.sql (runnable with psql -f) plus the GraphQL request body, and point to it from the README?
  2. Time the broad search too. Every generated title is "Synthetic Album N", so searching album matches all 100,000 rows, and relevance sort has to ts_rank and sort every one of them. That's our worst case and probably the first thing a grader types. Could you add that timing (relevance sort, first page) to the table?

Smaller notes, not blocking:

  • The README could say that if db:seed:large fails partway it should just be re-run, since the truncate and each batch commit separately and a crash leaves a partial catalog.
  • The ALBUM_COUNT=100000 pnpm db:seed:large example is Bash-only. The PowerShell form is $env:ALBUM_COUNT=100000; pnpm db:seed:large.
  • seedLarge.ts matches RETURNING rows to the catalog by index. Postgres keeps VALUES order in practice but doesn't promise it. Returning { id, title } and looking up by title would remove the assumption, since titles are unique.
  • The branch now conflicts with main in README.md and docs/AI-USAGE.md after #45, #47 and #49. It needs merging main in before it can go in.
  • I count 21 server tests, not 22.

evenkkl pushed a commit that referenced this pull request Oct 16, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
evenkkl pushed a commit that referenced this pull request Oct 16, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@rachelks
Copy link
Member

Looks good overall! 😊 Just one small issue: the Bash benchmark command in README line 107 uses double backslashes (\\) instead of single ones (\), so it won't run correctly. Could you fix that before merging?

Copy link
Member

@rachelks rachelks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last review-comment fixed! Approved!

@rachelks rachelks merged commit 9069bcb into main Oct 16, 2026
2 checks passed
@rachelks rachelks deleted the feat/7-large-scale-seed branch October 16, 2026 18:12
Sign in to join this conversation on GitHub.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Seed script and large-scale generator
3 participants